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FOREWORD 


The  Battlefield  Information  Systems  Technical  Area  is  concerned 
with  the  demands  of  the  future  battlefield  for  increased  man-machine 
complexity  to  acquire,  transmit,  process,  disseminate,  and  use  infor- 
mation. The  research  focuses  on  the  interface  problems  and  interactions 
within  command  and  control  centers  in  areas  such  as  topographic  products 
and  procedures,  tactical  symbology,  information  management,  user-oriented 
systems,  staff  operations  and  procedures,  and  integration  and  use  of 
sensor  systems. 

An  area  of  special  interest  is  that  of  problems  of  human  factors 
in  the  presentation  and  interpretation  of  surveillance  and  target  acqui- 
sition information.  One  relatively  new  source  of  intelligence  informa- 
tion is  remote  monitoring  of  the  battlefield  using  seismic,  acoustic, 
and  magnetic  unattended  ground  sensors.  When  these  remote  sensors  are 
activated  by  enemy  personnel  or  vehicle  movement,  a monitor  display  lo- 
cated behind  our  lines  indicates  the  activity.  From  this  display  the 
operator  can  derive  not  only  the  presence  of  the  enemy  but  the  direction 
and  speed  of  convoys  and  personnel,  the  number  of  vehicles  in  a convoy, 
and  the  composition  of  the  convoy,  for  example,  whether  it  consists  of 
armored  or  wheeled  vehicles. 

The  present  publication  reports  the  initial  investigation  of  oper- 
ator display  needs  for  unattended  ground  sensor  systems.  The  operational 
type  of  display,  the  RO  376,  event  versus  time  recorder,  was  compared  to 
two  types  of  situational  map  displays  and  found  to  be  superior.  (ARI 
Technical  Paper  281  compared  the  relative  effectiveness  of  four  differ- 
ent unattended  ground  sensors  in  terms  of  their  effects  on  monitor 
performance . ) 

This  research  was  done  in  cooperation  with  the  Combat  Surveillance 
and  Target  Acquisition  Laboratory  of  the  U.S.  Army  Electronics  Command 
at  Fort  Monmouth,  N.J.  It  is  responsive  to  requirements  of  Army  Project 
2Q662704A721  and  special  requirements  of  the  Office  of  the  Assistant 
Chief  of  Staff  for  Intelligence  and  the  Remotely  Monitored  Battlefield 
Sensor  System  Project.  The  cooperation  of  the  U.S.  Naval  Inshore  War- 
fare Command,  Atlantic,  and  of  LCDR  E.  Hockey  in  particular,  is  grate- 
fully acknowledged.  Without  this  excellent  support,  the  quality  and 
timeliness  of  this  experiment  would  have  been  seriously  impaired. 
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COMPARISON  OF  THREE  DISPLAY  DEVICES  FOR  UNATTENDED  GROUND  SENSORS 


BRIEF 


Requirement: 

The  experiment  was  designed  to  determine  the  relative  values  under 
typical  operating  conditions  of  three  methods  of  displaying  activations 
of  seismic  unattended  ground  sensors:  use  of  the  operational  RO  376 
event  recorder,  use  of  a situation  map  display,  and  use  of  time  compression 
with  the  situation  map  display. 

Procedure : 

Three  tape  recordings  lasting  2 hours  each  of  the  activations  of 
unattended  ground  sensors  were  compiled  from  the  data  bank  of  recordings 
of  unattended  ground  sensor  activations  taken  during  field  tests  under 
simulated  operational  conditions.  Typical  patterns  at  two  levels  of  tar- 
get activity  were  selected  to  include  both  personnel  and  vehicle  targets. 

To  provide  realistic  simulation,  recorded  activations  of  aircraft,  artil- 
lery, and  background  noise  likely  to  affect  the  interpretation  of  dis- 
plays were  included. 

Twelve  Naval  personnel  trained  and  experienced  in  the  use  of  the 
RO  376  were  given  4 hours  training  in  the  use  of  the  situation  map  dis- 
play and  the  display  used  with  time  compression.  Each  subject  then  in- 
terpreted each  of  the  three  displays  using  a different  set  of  recorded 
activations  each  time,  in  counterbalanced  order,  and  filled  out  a stan- 
dardized report  form.  The  reports  were  scored  for  number  of  correct  de- 
tections and  number  of  false  alarms  in  comparison  with  the  known  target 
activity  observed  in  the  Modern  Army  Selected  Systems  Test,  Evaluation, 
and  Review  (MASSTER)  tests  at  Fort  Hood,  Tex. 


Findings : 

Use  of  the  operational  RO  376  resulted  in  higher  accuracy  and 
greater  completeness  of  reports  than  did  use  of  the  other  displays. 

No  differences  were  found  between  the  situation  map  display  and  the 
situation  map  display  used  with  time  compression. 

Target  activity  level,  order  effects,  and  composition  of  the  taped 
activations  affected  operator  performance. 


Utilization  of  Findings: 

The  experiment  reported  here  was  an  early  effort  in  a series  to 
improve  the  interpretation  of  UGS  activations.  Because  of  its  superior- 
ity, the  RO  376  event  recorder  should  be  used  instead  of  the  situation 
map  display  in  interpreting  activations  of  sensors  deployed  in  strings. 

An  additional  study  that  would  deploy  sensors  using  the  area  in- 
trusion concept  of  sensor  deployment  ("grid"  or  "gated  array")  could 
be  an  additional,  useful  basis  for  evaluation. 

In  view  of  the  effect  on  performance  of  target  activity  level, 
order  effects,  and  composition  of  taped  activations,  these  conditions 
must  be  controlled  in  further  evaluations  of  ground  sensors  in  which 
operators  interpret  and  report  intelligence  information. 


COMPARISON  OP  THREE  DISPLAY  DEVICES  FOR  UNATTENDED  GROUND  SENSORS 


CONTENTS 


Page 

THE  DISPLAY  DEVICES  I 

The  Operational  Event  Recorder  1 

The  Situation  Map  Display  1 

OBJECTIVES  2 

METHOD  3 

Population  and  Sample  3 

Independent  Variables  3 

Dependent  Variables  4 

Experimental  Design  4 

Procedure  5 

RESULTS  6 

Wrongs  (False  Alarms) 6 

Detection- Rights  11 

Identification-Rights  14 

Detection  Accuracy  14 

CONCLUSIONS  AND  DISCUSSION  18 

Implications  of  the  Findings 20 

APPENDIX  A.  BRIEFING  FOR  EXPERIMENT  TO  EVALUATE  UGS  DISPLAYS  ...  21 

B.  TRAINING  OF  SUBJECTS  IN  THE  USE  OF  MAP  DISPLAYS  ...  25 

C.  TARGET  REPORT  FORM 27 

DISTRIBUTION  LIST 29 

LIST  OF  TABLES 

Table  1.  Target  distribution  6 


2.  Analysis  of  variance  table  for  Detection  Wrongs- 
False  Alarms  


7 


Table  3.  Average  number  of  Wrongs  by  Day  and  Display 


9 


4.  Number  of  possible  false  alarms  (noise)  for 

sections  of  scenarios  10 

5.  Analysis  of  variance  table  for  Detection- Rights  12 

6.  Mean  number  of  right  detections  per  hour  for  the 

Displays  and  Activity  levels  13 

7.  Mean  number  of  right  detections  per  2-hour  period 

for  Displays  and  Day 13 

8.  Analysis  of  variance  table  for  Identifications  15 

9.  Analysis  of  variance  table  for  Accuracy  16 

10.  Mean  accuracy  for  Displays  and  Experience  level  17 

11.  Mean  accuracy  by  Displays  and  Day 18 


LIST  OF  FIGURES 

Figure  1.  Greco-Latin  square  design  5 


COMPARISON  OF  THREE  DISI’l AY  DEVICES  FOR  UNATTENDED  GROUND  SENSORS 


'I  Vo  basic  type:;  of  unattended  ground  sensor  (UGS)  displays  .ire  be- 
ing considered  for  major  use  by  the  Army:  the  operational  event  versus 
time  (XT)  recorder  and  the  situation  map  display.  The  evaluation  of 
these  devices  under  operational  conditions  was  an  early  step  in  a pro- 
gram seeking  to  improve  operator  performance  in  interpreting  and  re- 
porting displays  of  UGS  activations. 


THE  DISPLAY  DEVICES 
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! 
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The  Ope t at  i one  I Evenj  Keyoidei 

The  two  operational  forms  of  the  XT  recorder  are  the  30-pen  RO  376 
event  recorder  and  the  BASS  III  60-pen  recorder.  These  devices  portray 
l ime  vertically  and  denote  the  activations  of  sensors  in  the  field  by 
short  horizontal  lines  in  specified  columns;  each  column  is  associated 
with  the  return  from  a single  sensor.  The  resulting  permanent  record, 
plus  information  about  where  the  strings  of  three  or  more  sensors  are 
deployed,  how  far  apart  they  are,  and  what  type  of  sensor  is  being  used, 
enables  the  operator  to  make  one  or  more  of  the  following  estimations: 

1.  when  a possible  enemy  activates  a sensor, 

2.  how  many  and  what  kind  of  enemy  object:;  are  present  (person- 
nel, vehicles,  or  aircraft), 

3.  where  they  are, 

4.  what  direction  they  are  going,  and 

5.  what  speed . 

Because  the  record  is  permanent,  the  operator's  report  can  be  checked. 
Likewise,  the  battalion  intelligence  officer  (S2)  or  others  in  similar 
operations  positions  can  review  fairly  quickly  the  type  of  action  that 
has  occurred  in  the  past  few  hours. 


The  Situation  Map  Display 

The  situation  map  display  consists  of  a map  placed  over  small 
lights  that  correspond  to  the  location  of  a sensor  in  the  field.  The 
light  blinks  rapidly  several  times  for  each  activation  of  its  associ- 
ated sensor  and  then  goes  off  until  the  next  sensor  activation.  (Most 
sensors  have  a short  rest  period  of  4 to  10  seconds  during  which  they 
cannot  be  reactivated.)  At  present,  prototype  30  and  120  sensor  displays 
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have  been  constructed.  There  is  no  permanent  record  of  the  activations 
with  these  devices  except  by  the  proposed  addition  of  a tape  recorder 
or  similar  device  to  record  incoming  signals.  Using  a tape  recorder 
would  make  it  possible  to  resend  the  signals  to  the  map  display  and 
reactivate  the  lights. 

An  advantage  of  the  situation  map  display  is  that  it  helps  the 
operator  quickly  identify  the  location  and  direction  of  movement  of  the 
source  of  activations  by  direct  simultaneous  observation  of  the  map  and 
activations.  A potential  advantage  of  this  system  is  the  capability  to 
review  activations  in  compressed  time  by  playing  the  proposed  tape  re- 
cording at  a higher  speed.  Target  activations  that  would  be  missed  be- 
cause of  noise  or  slow  velocity  of  the  target  past  the  sensors  might  be 
discovered  more  easily  in  time  compression. 

Similarly,  if  only  the  last  sensor  or  the  last  two  sensors  gives 
strong  returns,  playback  showing  the  first  couple  of  sensors  in  the 
string  with  knowledge  that  the  last  returns  are  strong  may  be  helpful 
(although  suggestion  effects  also  may  cause  false  alarms).  In  addition, 
someone  like  the  S2  could  quickly  see  the  entire  situation  developing 
in  the  context  of  the  map  with  all  its  information,  including  location 
of  defense  positions,  supplies,  and  reserves.  He  could  very  qu  ckly 
review  what  had  occurred  in  the  last  several  hours  and  determine  the 
pattern  and  perhaps  the  enemy's  intention. 

Displays  of  both  types  have  been  proposed  for  future  operational 
use.  A comparison  of  the  two  types  in  a variety  of  situations  and  a 
determination  of  the  usefulness  of  a time-compression  aid  with  the 
situation  map  display  are  requirements  of  one  in  a series  of  research 
projects  concerning  factors  affecting  the  performance  of  operators 
monitoring  remote  sensing  devices.  Although  the  value  of  the  RO  376 
event  recorder  has  been  established  experimentally,  the  value  of  the 
situation  map  display  and  of  the  situation  map  display  with  time  com- 
pression has  not. 


OBJECTIVES 

1.  To  determine  the  effectiveness  of  time  compression  as  a dis- 

* play  technique  by  comparing,  under  typical  operating  conditions,  the 

relative  utility  of  the  RO  376  plotter,  the  situation  map  display,  and 
the  situation  map  display  using  the  time-compression  technique. 

2.  To  obtain  estimates  of  operator  performance  indexes  using  the 
RO  376  and  the  situation  map  display  under  a variety  of  operational 
conditions. 
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METHOD 


Population  and  Sample 

Army  enlisted  operators  who  have  been  school  trained  at  Fort 
Huachuca  and  who  have  had  some  field  experience  constitute  the  popula- 
tion of  concern.  Because  of  the  difficulty  in  obtaining  a sufficiently 
large  sample  of  experienced  Army  personnel  at  one  location,  12  Navy 
personnel  were  used;  of  these,  10  were  school  trained  and  had  field  ex- 
perience in  several  maneuvers.  Two  had  had  on-the-job  training  and  no 
experience.  All  subjects  were  given  4 hours  of  training  on  the  use  of 
the  situation  map  display  and  compressed  time.  In  addition,  the  2 in- 
experienced subjects  were  given  a review  on  the  use  of  the  RO  376 
recorder. 


Independent  Variables 

1.  Displays . The  three  displays  were  the  RO  376  event  recorder, 
the  situation  map  display,  and  the  situation  map  display  used  with 
time  compression. 

2.  Experience . Each  subject  was  ranked  by  the  commanding  officer 
on  amount  of  experience  in  reading  out  UGS  displays.  The  subjects  were 
then  separated  into  a high  group  and  a low  group.  The  rankings  were 
checked  against  self-report  data.  The  high  group  averaged  25  weeks  of 
total  experience,  and  the  low  group  averaged  only  8 weeks.  The  high 
group  also  averaged  4.8  weeks  of  participation  in  UGS  exercises;  the 
low  group  averaged  only  2.6  weeks. 

3.  Scenario.  Three  2-hour  scenarios  were  developed;  each  scenario 
was  divided  into  two  30-minute,  high-activity  sections  and  two  30-minute, 
low-activity  sections.  Each  section  contained  both  walking  personnel 
and  vehicular  targets. 

4.  Periods . Each  operator  worked  for  12  30-minute  periods.  This 
variable  has  no  intrinsic  meaning.  In  the  analysis,  however,  it  was 
divided  into  Time,  Day,  and  Activity  variables. 

5.  Time . The  first  hour  of  a scenario  versus  the  second  hour — a 
measure  of  short-term  time  effects. 

6.  Day.  The  first  scenario  given  versus  the  last  scenario  given 
(the  two  on  different  days) — a measure  of  longer  term  time  effects. 

7.  Target  Activity  Level.  The  six  low-target  activity  sections 

of  the  scenarios  (2  to  3 targets  per  30  minutes)  were  compared  to  the 

six  high-target  activity  sections  (6  to  8 targets  per  30  minutes). 

8.  Order.  The  three  groups  of  four  operators  used  the  displays 

and  scenarios  in  a different  order. 
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Impendent  Variables 


1.  Detection- Rights . 1 f an  operator  reported  a target  at  the 

time  a target  was  activating  the  designated  pens  of  the  display,  the 
response  was  classified  as  a Detection- Hi ght . Normally,  this  variable 
is  perfectly  correlated  with  the  detection  completeness  of  a report . 
Completeness  is  a ratio  (expressed  as  a percentage)  of  targets  found  to 
total  number  of  targets  present.  The  simpler  measure,  "Rights,"  gen- 
erally was  used  because  it  is  easier  to  compute  and  is  readily  convert- 
ible to  completeness.  However,  completeness  figures  had  to  be  used 
with  the  Activity  variable  because  there  are  more  targets  available 

for  detection  in  the  high-activi ty  level  than  iri  the  low. 

2.  Identi fication- Rights . If,  in  addition  to  the  above,  the 
operator  classified  the  target  correctly  by  type  (personnel  or  vehicle), 
the  response  was  also  classified  as  an  Identification-Right. 

3.  Detection  Wrongs- False  Alarms.  If  an  operator  reported  a 
target  at  a time  when  no  target  was  causing  an  activation  on  the  desig- 
nated pens  of  the  display,  the  response  was  classified  as  a Wrong-False 
Alarm. 

4.  Accuracy- De tcction.  This  was  the  ratio  (expressed  as  a per- 
centage) of  the  number  of  Detection-Rights  to  the  sum  of  the  number  of 
Detection- Rights  and  Wrongs  (or  total  number  of  responses  an  operator 
made ) . 


Experimental  Design 

A three-by- three  replicated  Greco-Latin  square  design  (Figure  1) 
was  used.  Of  primary  interest  were  differences  in  the  main  effect 
(the  display  used)  and  its  interactions  with  activity  level  (high  or 
low)  and  field  experience  (high  or  low) . An  analysis  of  variance  was 
computed  to  test  for  differences  because  of  Displays  (3) , Order  (3) , 
Experience  (2),  Period  (12),  Activity  Level  (2),  Time  (2),  Day  (2), 
Scenarios  (3);  and  the  interactions  of  devices  with  Activity  Level, 
Day,  Time,  and  Experience,  and  of  Order  with  Experience.  Other  simple 
interactions,  such  as  Scenarios  and  Day,  and  higher  order  interactions 
were  not  specifically  analyzed  but  lumped  into  a residual  term.  In 
addition,  Dunn's  tests^  were  used  for  testing  the  significance  of  dif- 


1st  Test 

2nd  Test 

3rd  Test 

Session 

Session 

Session 

Group 

1 

C-Map  display  with 

D-Map  display 

R-RO  376 

compressed  time 

Group 

2 

D-RO  376 

B-Map  display  with 

C-Map  display 

compressed  time 

Group 

3 

B-Map  Display 

C-RO  376 

D-Map  display  with 

compressed  time 

No  te : 

B 

, C,  and  D refer  to 

the  three  2-hour  scenarios. 

Fiqure  1.  Greco-Latin  square  design. 


Procedure 

Three  Uher*  tape  recorders  and  interface  units  were  used  to  drive 
the  RO  376  and  two  30-light  situation  map  displays.  The  Uher  recorders 
are  capable  of  being  played  at  speeds  ranginq  from  recording  speed  to 
eight  times  recording  speed,  thereby  providing  means  for  "sending"  to 
the  display  the  recorded  activations  for  all  conditions  of  the  experiment. 
Each  tape  recorder  was  paired  with  a single  display  to  permit  the  flexi- 
bility required  by  the  experimental  design. 

The  scenarios  were  developed  from  material  obtained  by  taping  typi- 
cal activations  produced  by  personnel,  armored  vehicles,  aircraft,  artil- 
lery, and  noise  during  Project  .1.030  and  BASS  III  tests  at  MASSTER,  Fort 
Hood,  Tex.  Because  these  were  controlled  exercises,  target  location  and 
time  were  known  and  could  be  related  to  sensor  activations.  Typical  noise 
such  as  that  caused  by  weather,  noisy  sensors,  artillery,  and  aircraft 
were  not  eliminated  and  were  added  specifically  in  particularly  unevent- 
ful portions  of  the  performance  measures.  Portions  of  the  above  tapes 
showing  "good"  target  returns  were  selected  and  combined  into  three 
2-hour  tapes  (three  scenarios) . These  tapes  were  played  back  to  acti- 
vate the  displays  to  reproduce  conditions  as  they  existed  in  the  field. 

As  stated  in  defining  the  Scenario  variable,  each  of  the  2-hour 
tapes  was  designed  to  have  two  30-minute  portions  displaying  a high-target 
activity  level  and  two  portions  displaying  a low-tarqet  activity  level . 

A target  is  defined  as  a group  of  one  to  nine  personnel  or  one  to  six 


Commercial  designations  are  given  only  in  the  interest  of  precise  de- 
scription; their  mention  does  not  constitute  endorsement  by  the  Army. 


vehicles  passing  by  arte}  activating  the  ent  ire  string  of  sensors.  The 
number  of  targets  per  scenario  and  time  period  is  shown  in  Table  1. 


Table  1 

Target  Distribution 


Scenario 

30- 

Targets  per 
-minute  period 

Number  of  targets 

1st 

2nd 

3rd 

4 th 

Personnel 

Vehicle 

Tc 

ital 

B 

3 

8 

2 

7 

9 

11 

20 

c 

2 

6 

2 

7 

7 

10 

17 

D 

2 

6 

2 

7 

7 

10 

17 

Three  subjects  at  a time  were  given  a general  briefing  explaining 
the  purpose  of  the  research  and  the  role  of  the  subjects  (Appendix  A) . 

The  subjects  were  then  given  4 hours  of  instruction  and  practice  on  a 
situation  map  display  and  use  of  compressed  time  (Appendix  B) . If  neces- 
sary, a review  of  the  use  of  the  RO  376  also  was  given. 

Each  subject  was  assigned  to  one  of  the  three  display-scenario 
stations  for  the  first  2 hours.  After  a break  (1  hour  or  overnight) , 
the  subject  went  to  another  display-scenario  station  for  the  second 
2 hours.  Following  another  similar  break,  each  subject  was  assignor)  to 
the  last  display-scenario  station.  The  subjects  recorded  target  informa- 
tion on  the  report  form  shown  in  Appendix  C.  Their  responses  were  scored 
against  known  target  information  to  derive  the  different  scores  for  the 
analyses. 


RESULTS 


Wrongs  (False  Alarms) 

Table  2 shows  the  analysis  of  variance  results  for  Wrongs.  Of 
primary  interest  are  the  statistically  significant  differences  found 
on  the  Displays  variable.  The  average  number  of  false  alarms  a man  a 
2-hour  period  was  RO  376,  5.6;  map  display,  9.8;  and  map  display  with 
TC,  12.6. 


; 


■ 
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Table  2 


Analysis  of  Variance  Table  for  Detection 
Wrongs-False  Alarms 


Significance 


Variable 

df 

SS 

MSS 

F ratio 

level 

Experience 

1 

.4444 

.4444 

.0358 

NS 

Order  effects 

2 

127. 7917 

63.8958 

5.1460 

05 

Experience 

x order 

2 

82.9306 

41.4653 

3.3395 

NS 

Subject  within  groups 

6 

74.5000 

12.4167 

Displays 

2 

74.6250 

37. 3125 

15.9394 

01 

Displays  x 

activities 

2 

2.9305 

1.4652 

.6259 

NS 

Displays  x 

experience 

2 

13.4306 

6. 7153 

2.8687 

NS 

Displays  x 

time 

2 

10.0139 

5.0070 

2.1389 

NS 

Displays  x 

day 

2 

65.3333 

32.6667 

13.9548 

01 

Periods 

11 

85.6667 

7.7879 

3.3269 

01 

(Activity) 

(1) 

(18.7778) 

18.7778 

8.0216 

01 

(Time) 

(1) 

(17.3611) 

17.3611 

7.4164 

01 

(Day) 

(1) 

(10.0104) 

10.0104 

4. 2763 

05 

Scenarios 

2 

12.7917 

6.3958 

2.7322 

NS 

Residual 

43 

175.0416 

4.0707 

1.7389 

05 

Error 

66 

154.5000 

2.3409 

Total 


143  880.0000 


The  difference  between  the  RO  376  display  and  the  average  of  the 
other  two  displays  (tested  by  Dunn's  test)  is  statistically  significant 
at  the  .OS  level.  Similar  comparison  of  the  map  display  versus  the  map 
display  with  time  compressions  shows  no  significant  difference.  Thus, 
on  the  average,  twice  as  many  false  alarms  occurred  using  the  map  dis- 
play as  compared  to  using  the  RO  376. 

The  interaction  between  Displays  and  Activity  Level  was  not  statis- 
tically significant.  Thus,  the  difference  found  between  Displays  was  not 
restricted  to  either  low-  or  high-target  activity,  but  can  be  generalized 
to  both . 

Similarly,  no  significant  difference  was  found  for  the  interactions 
of  Experience  and  Display.  It  thus  appears  that  the  difference  in  dis- 
plays can  be  generalized  over  experience.  However,  contrary  to  expec- 
tations, a significant  difference  in  performance  was  not  found  between 
the  high-  and  low-experience  groups.  Several  factors  might  explain  this 
finding;  for  example,  that  all  subjects  were  "sufficiently"  experienced 
to  be  relatively  homogeneous  in  performance.  However,  since  experience, 
in  general,  should  improve  performance,  the  finding  is  suspect,  and  gen- 
eralizations regarding  experience  should  be  restricted  to  the  sample 
used  in  the  present  study. 

The  interaction  of  Displays  and  Time  (first  hour  versus  second  hour 
of  a scenario)  was  not  statistically  significant  at  the  .05  level.  Thus, 

I the  differences  found  between  Displays  are  not  restricted  to  either  the 

first  or  second  hour  of  a scenario  but  can  be  generalized  over  the  com- 
plete 2-hour  period  and  any  time  effect  therein. 

To  summarize,  interactions  of  Displays  with  Activity  Level,  Experi- 
ence, and  Time  were  not  statistically  significant.  This  result  indicates 
that  the  differences  found  in  Displays  can  be  generalized  over  Activity 
Level,  Experience,  and  Time. 

, The  interaction  between  Displays  and  Day  was  statistically  signi- 

ficant at  the  .01  level.  Table  3 shows  the  mean  number  of  Wrongs  per 
2 hours  by  Day  and  Display. 

Krom  Table  3,  it  seems  clear  that  the  advantage  of  the  RO  376  dis- 
i play  shown  earlier  occurred  over  both  days.  However,  the  interaction 

f effect  apparently  indicates  that  use  of  the  map  display  without  compressed 

, time  resulted  in  a decrease  in  Wrongs  from  Day  1 to  Day  2,  possibly  due 

to  a learning  effect,  whereas  use  of  the  other  two  displays  showed  a 
decrement  in  performance  from  Day  1 to  Day  2.  Performance  on  the  map 
display  with  time  compression  in  particular  exhibited  twice  as  many 
Wrongs  on  Day  2.  A reason  for  this  interaction  such  as  fatigue,  differ- 
ential learning,  or  experience  factors  does  not  appear  reasonable, 
f Further  examination  of  this  finding  seems  indicated.  This  particular 

; finding  does  not  affect  the  results  or  the  interpretation  of  main  effect 

differences  for  the  Display  variable  because  use  of  the  RO  376  resulted 
in  fewer  Wrongs  on  both  days. 
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Table  3 


Average  Number  of  Wrongs 


a 


by  Day  and  Display 


Map  display 

RO  376 

Map  display 

with  time  compression 

Day  1 

3.2 

13.2 

10.2 

Day  2 

4.5 

8.5 

21.5 

cl 

These  means  cannot  be  compared  to  previously  given  means  for  displays 
because  they  are  based  on  a smaller  group. 


Statistically  significant  effects  were  found  for  Period.  The 
Period  variable  is  simply  the  12  30-minute  periods  (4  30-minute  periods 
per  2-hour  scenario)  used  in  the  experiment.  Period  could  be  interpreted 
as  some  combination  of  scenario,  time,  fatigue,  and  learning  effects,  and 
thus  is  of  no  value  by  itself. 

The  three  major  components  of  Period  (Activity,  Time,  and  Day) 
that  were  thought  to  have  most  significance  to  this  research  were  ana- 
lyzed further.  Table  2 shows  a significant  difference  on  all  three  vari- 
ables. The  high-activity  condition  resulted  in  fewer  wrong  responses 
(3.9  per  hour)  than  the  low-activity  condition  (5.4  per  hour).  Thus,  it 
could  be  theorized  that  during  high  activity  the  operator  makes  fewer 
mistakes  because  he  is  too  busy.  However,  activity  effects  could  be 
confounded  with  a differential  difficulty  level  of  each  part  of  a scenario. 

Different  sets  and  arrangements  of  activations  were  present  during 
each  30  minutes  of  any  one  scenario.  Thus,  the  low-activity  parts  of 
the  scenario  could  contain  more  error-producing  noise.  All  scenarios 
were  analyzed  with  respect  to  the  amount  of  noise  present  in  each 
30  minutes  that  would  cause  an  operator  to  report  a target.  Any  noise 
generated  by  aircraft,  artillery,  environment,  etc.,  that  caused  two 
sensors  in  a string  to  activate  within  a short  period  of  time  was  defined 
as  a "possible  false  alarm."  The  number  of  possible  false  alarms  was 
used  as  the  measure  of  amount  of  noise. 

Table  4 indicates  that  a differential  amount  of  noise  could  cause 
the  differences  found  in  the  Activity  variable.  There  were  60  possible 
false  alarms  in  the  low-activity  section  but  only  39  in  the  high.  Thus, 
there  are  at  least  two  possible  explanations:  either  the  high-activity 
level  prevented  mistakes  by  keeping  the  operator  busy  or  noise  differ- 
ences between  the  high-  and  low-activity  sections  of  the  scenarios  caused 
the  differences. 
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Tab le  4 


Number  of  Possible  False  Alarms  (Noise)  for 
Sections  of  Scenarios 


Scenario 

Low  activity 

High  activity 

Tota  1 

B 

15 

12 

27 

C 

16 

16 

32 

D 

29 

11 

40 

Total 

60 

39 

1st  hour 

2nd  hour 

B 

13 

14 

27 

C 

17 

15 

32 

D 

20 

20 

40 

Total 

50 

49 

The  significant  time  difference  (Table  2),  that  is,  the  first  hour 
of  a scenario  versus  the  second  hour,  contraindicates  a fatigue  effect. 

An  average  of  5.4  wrong  responses  were  made  the  first  hour  as  opposed 
to  4.0  the  second  hour.  However,  the  second  hour  of  each  of  the  scenarios 
could  have  contained  more  noise  and  thus  have  caused  this  effect. 


Table  4 indicates  that  the  first  hour  and  second  hour  of  each 
scenario  were  about  equal  in  the  amount  of  noise  that  could  have  caused 
false  alarms  (50  and  49) . Another  possible  explanation  could  be  learning, 
except  that  there  was  no  feedback  as  to  the  correctness  of  each  report 
an  operator  made.  Thus,  learning  was  made  very  difficult.  Also,  the 
analysis  of  the  variable  Day  (Day  1 versus  Day  2)  did  not  indicate  a 
consistent  learning  effect.  If  the  above  difference  can  be  shown  in 
other  situations,  it  could  mean  that  the  operator  can  spend  longer  peri- 
ods of  time  interpreting  displays  without  experiencing  a decrement  in 
one  aspect  of  performance  (i.e.,  increasing  Wrongs)  due  to  fatigue. 

This  effect  will  be  shown  again  under  the  discussion  of  the  Accuracy 
variable. 


The  significant  difference  (Table  2)  found  between  Day  1 (4.5  Wrongs 
per  hour)  and  Day  2 (5.8  Wrongs  per  hour)  contraindicates  learning  ef- 
fects in  general  (see  previous  discussion  on  Display-Day  interaction) . 
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Assuming  th.it  the  Day  effect  is;  riot  long  lasting  but  only  an  experimental 
artifact,  future  DCS  experimentation  mur;t  control  for  this  variable. 

The  analysis  of  variance  (Table  2)  shows  no  significant  differences 
between  scenarios,  indicating  that  the  stimulus  material  used  in  each 
scenario  was  sufficiently  similar  as  to  produce  equal  numbers  of  false 
alarms.  Because  of  the  observed  noise  differences  between  scenarios 
(Table  4),  additional  analyses  (orthogonal  comparisons)  were  computed 
between  Scenario  D and  the  average  of  Scenarios  B and  C and  between 
Scenarios  B and  C.  No  significant  differences  between  the  means  were 
found . 

The  significant  order  effect  (Table  2)  indicates  that  the  order  of 
presentation  of  the  combination  of  Scenario  and  Display  affected  per- 
formance. Future  research  to  improve  performance  on  unattended  ground 
sensors  should  control  this  variable. 


Detect ion- Rights 

Table  5 presents  the  analysis  of  variance  results  for  Detection- 
Rights.  Of  most  interest  is  the  significant  difference  (.05  level)  found 
for  the  Display  variable.  Use  of  the  RO  376  event  recorder  resulted  in 
the  highest  average  number  of  rights  for  the  2-hour  period  (15.8)  and  use 
of  the  map  display  alone,  the  lowest  (14.2).  Dunn's  test  comparing  the 
RO  376  with  the  two  map  displays  indicated  a significant  difference  at 
the  .01  level.  The  same  test  comparing  the  two  map  displays  showed  no 
significant  difference. 

The  significant  (.05)  interaction  of  Displays  and  Activity  is  of 
interest  because  it  shows  the  difference  found  in  the  Displays  variable 
occurred  in  the  high-activity  condition  only  (Table  6) . Table  6 also 
shows  the  corresponding  completeness  values  in  parentheses.  Under  the 
low-activity  conditions,  all  displays  were  about  equal  (90%).  However, 
the  high-activity  conditions  resulted  in  lowering  completeness  to  only 
77%  for  the  map  display  and  map  display  with  time  compression.  Thus, 
in  the  more  difficult  situation  there  was  a decrement  in  performance 
with  these  displays. 

The  display  by  experience  and  the  display  by  time  interactions 
yielded  no  significant  effects.  Thus,  as  with  the  Wrongs  analysis, 
no  differences  due  to  experience  and  time  were  found  with  the  sample 
used  in  the  present  experiment. 
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Table  5 


Analysis  of  Variance  Table  for  Detection- Rights 


Variable 

df 

SS 

MSS 

F ratio 

Significance 

level 

Exper ience 

1 

1.3611 

1.3611 

.8099 

NS 

Order  effects 

2 

.2916 

.1458 

.0868 

NS 

Experience  x order 

2 

.5139 

.25695 

.1529 

NS 

Subject  within  groups 

6 

10.0834 

1 . 6806 

Displays 

2 

3.8750 

1.9375 

3.7702 

05 

Displays  x activities 

2 

4.2639 

2.13195 

4. 1486 

05 

Displays  x experience 

2 

.2639 

. 13195 

.2568 

NS 

Displays  x time 

2 

1.3472 

.6736 

1.3108 

NS 

Displays  x day 

2 

7. 5209 

3 . 7604 

7.3174 

01 

Periods 

11 

458.0833 

41.6489 

81.0350 

01 

(Activity) 

(1) 

(434.0278) 

434.0278 

844. 5764 

01 

(Time) 

(1) 

1.7778 

1.7778 

3.4594 

NS 

(Day) 

(1) 

1 . 7604 

1.7604 

3.4256 

NS 

Scenarios 

2 

12. 1667 

6.08335 

11.8376 

01 

Residual 

43 

40.0625 

. 931'* 

1.8130 

05 

Error 

66 

33.9166 

.5139 

Total 

143 

573.7500 
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Table  6 


Mean  Number  of  Right  Detections  per  Hour  for  the 
Displays  and  Activity  Levels 


Map  display  with 


Level 

RO  376 

Map  display 

time  compression 

High  activity 

11.9  (87) 3 

10.3  (76) 

10.5  (77) 

Low  activity 

3.9  (90) 

3.9  (90) 

4.0  (92) 

a 

Completeness 

(100  x Rights  divided 

by  total  number  of 

targets) . 

A significant  difference  was  found  for  the  interaction  of  Displays 
and  Day.  Table  7 gives  the  mean  number  of  Detection-Rights  for  each 
display  for  each  day.  The  superiority  of  the  RO  376  is  clearly  associ- 
ated with  whatever  the  Day  effect  represents.  Day  1 performance  with  the 
RO  376  was  about  equal  to  that  for  the  other  displays.  On  Day  2,  the 
RO  376  was  clearly  superior.  The  reason  for  this  effect  is  hard  to  de- 
termine, but  one  hypothesis  is  related  to  pretest  training.  Because  all 
subjects  had  some  practical  experience  in  using  the  RO  376  display  on 
field  exercises,  essentially  no  pretest  training  was  given  on  the  RO  376; 
all  available  time  was  devoted  to  the  map  displays  and  time  compression. 
Conceivably , a warmup,  acclimatization,  or  practice  was  needed  to  bring 
the  subjects  up  to  their  normal  efficiency  on  the  RO  376. 

Table  7 

Mean  Number  of  Right  Detections  per  2-Hour 
Period  for  Displays  and  Day 


Day 

RO  376 

Map  display 

Map  display  with 
time  compression 

Day  1 

13.8 

15.5 

14.5 

Day  2 

18.0 

15.0 

14.0 

The  significant  difference  found  for  Period  (Table  5)  could  be  at- 
tributable to  various  time  or  scenario- related  variables.  Three  vari- 
ables (Activity,  Time,  and  Day)  were  selected  as  relevant  to  this  study. 

A major  cause  of  the  difference  found  was  the  Activity  variable.  The 
significant  difference  found  for  the  Activity  variable  was  expected, 
because  there  are  many  more  targets  in  the  high-activity  section  of  the 
scenario  than  in  the  low-activity  section.  No  significant  difference 
was  found  for  either  the  Time  or  Day  variable. 

The  difference  found  for  the  scenarios  was  to  be  expected,  and  the 
design  of  the  experiment  took  this  prospect  into  account  in  order  not  to 
confound  other  variables.  The  differences  in  Rights  are  attributable 
to  differences  in  number  of  targets  in  each  scenario  and  to  differences 
in  target  difficulty. 

Order  effects  were  not  significant.  Neither  the  order  of  stimulus 
material  nor  that  of  the  displays  used  affected  performance. 

Identification- Rights 

Table  8 presents  the  analysis  of  variance  results  for  Identification- 
Rights.  No  significant  difference  was  found  for  the  Displays  variable. 

Use  of  the  RO  376  event  recorder  resulted  in  14.2  correct  identifications 
for  the  2-hour  period;  use  of  the  map  display  with  and  without  time  com- 
pression resulted  in  13.2  and  13.3  correct  identifications,  respectively. 
Dunn's  test  indicated  no  significant  difference  between  the  RO  376  anr’ 
the  average  of  the  two  map  displays  and  between  the  two  map  displays. 

These  results  are  quite  different  from  those  for  Detection-Rights. 
Apparently,  the  RO  376  made  it  possible  for  the  operator  to  detect  more 
targets  than  with  the  other  displays.  There  was,  however,  no  significant 
advantage  among  displays  in  identifying  targets.  Except  for  these  dif- 
ferences, the  analyses  of  variance  for  Detection-  and  Identification- 
Rights  (Tables  5 and  8)  gave  similar  results.  The  preceding  discussion 
for  Detection-Rights  applies  also  to  the  Identification-Rights. 


Detection  Accuracy 

Table  9 shows  the  results  of  the  analysis  for  detection  accuracy 
(100  x Rights,  divided  by  the  total  number  of  targets  reported) . The 
average  accuracy  of  the  operator's  reports  was  80.1%  using  the  RO  376, 
64.8%  using  the  map  display,  and  58.8%  using  the  map  display  with  time 
compression.  Dunn's  test  comparing  the  RO  376  to  the  average  of  the 
other  two  showed  a significant  difference  at  the  .01  level.  The  same 
test  comparing  the  two  map  displays  was  not  significant. 
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Table  8 


Analysis  of 

Variance  Table 

for  Identif 

ications 

Signi ficance 

Variable 

df 

SS 

MSS 

F ratio 

1 eve  1 

Experience 

1 

6.2500 

6.2500 

.8287 

NS 

Order  effects 

2 

1 .3472 

.6736 

. 0893 

NS 

Experience  x order 

2 

2.0417 

1 .0208 

. 1354 

NS 

Subject  within  qroups 

6 

45.2500 

7.5417 

Displays 

2 

1.7222 

.8611 

.9057 

NS 

Displays  x activities 

2 

2.3889 

1.1944 

1.2562 

NS 

Displays  x experience 

2 

.1667 

. 0834 

.0877 

NS 

Displays  x time 

2 

1.0555 

.5278 

.5551 

NS 

Displays  x day 

2 

9.5209 

4.7604 

5.0067 

01 

Periods 

11 

387. 555 

35.2323 

37.0554 

01 

(Activity) 

(1) 

(354.6944) 

354.6944 

373 .0484 

01 

(Time) 

(1) 

(2.7778) 

2.7778 

2.9215 

NS 

(Day) 

(1) 

(1.0416) 

1.0416 

1.0955 

NS 

Scenarios 

2 

11.5139 

5. 7570 

6.0549 

01 

Residual 

43 

48.6597 

1.1316 

1. 1902 

NS 

Error 

66 

62.7500 

.9508 

Total 

143 

580.2222 
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Analysis  of  Variance  Table  for  Accuracy 
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The  interaction  between  Displays  and  Activity  was  not  significant, 
indicating  that  the  differences  between  displays  can  be  generalized 
across  activity  levels.  However,  the  interaction  of  Displays  with  Ex- 
perience was  significant  at  the  .01  level.  Table  10  presents  the  mean 
accuracy  by  display  and  by  experience  level.  The  high-experience  group 
obtained  a higher  mean  accuracy  than  the  less-experienced  group  when  using 
the  map  display  and  map  display  with  time  compression.  The  less-experienced 
group,  however,  had  a higher  mean  accuracy  when  using  the  RO  376.  This 
inconsistency  again  casts  doubt  on  the  nature  of  the  Experience  variable 
because,  in  general,  more  experience  should  result  in  better  performance, 
especially  on  the  RO  376,  the  device  on  which  the  "experienced"  operators 
would  have  had  experience.  When  experience  alone  was  considered,  however, 
the  analysis  of  variance  showed  no  difference  between  the  two  groups, 
according  to  the  findings  for  Wrongs,  Detection-Rights,  and  Identification- 
Rights  . 


Table  10 


Mean 

Accuracy 

for  Displays  and  Experience 

Level 

Level 

RO  376 

Map  display 

Map  display  with 
time  compression 

High 

experience 

72 

68 

63 

Less 

experience 

87 

62 

55 

The  significant  interaction  (Table  9)  between  Display  and  Day  is 
predictable  from  the  results  of  Wrongs  and  Rights  analyses.  Table  11 
exhibits  these  mean  accuracy  results  with  slightly  better  performance 
on  the  second  day  for  the  RO  376  and  map  display,  and  poorer  performance 
on  Day  2 for  the  map  display  with  time  compression.  The  difference  on 
the  RO  376  from  Day  1 to  Day  2 is  probably  due  to  scenario  differences, 
but  the  other  differences  cannot  be  explained  in  this  way.  Further  ex- 
perimentation on  this  differential  effect  is  needed. 

The  significant  effect  found  for  the  Periods  variable  is  attribut- 
able to  the  Activity  and  Time  variables  that  were  significant  at  the 
.01  and  .05  levels,  respectively.  The  mean  accuracy  found  for  the  high- 
activity  condition  was  77%  as  compared  to  the  mean  for  the  low-activity 
condition  of  59%.  The  lower  accuracy  found  for  the  low-activity  condi- 
tions might  appear  to  contradict  logic  because  the  operator  has  more 
time  during  the  low-activity  conditions.  However,  this  result  may  be 
largely  attributable  to  a greater  number  of  targets  presented  in  the 
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hiqh-activity  section  of  the  scenarios  and  the  greater  number  of  possible 
false  alarms  due  to  noise  in  the  low-activity  sections  (see  Table  4). 


Table  11 

Mean  Accuracy  by  Displays  and  Day 


Map  display  with 

Day 

RO  376 

Map  display 

time  compression 

Day  1 

83 

61 

58 

Day  2 

88 

69 

41 

The  significant  time  effect  shows  that  overall  performance  improved 
from  64%  during  the  first  hour  of  a scenario  to  71%  for  the  last  hour. 
This  change  is  largely  due  to  a decrease  in  wrong  response  because  there 
were  no  significant  differences  for  the  Rights  variable. 

The  difference  found  for  the  scenarios  was  expected  and  is  a reflec- 
tion of  the  differences  found  for  the  Detection-Rights  variable  due  to 
differences  in  number  and  difficulty  of  the  targets  in  each  scenario. 

The  experiment  was  designed  to  control  these  differences  in  order  not  to 
confound  other  variables  and  ta  increase  the  sensitivity  of  the  analysis. 


CONCLUSIONS  AND  DISCUSSION 

The  data  concerning  the  Display  variable  are  consistent.  Use  of 
the  RO  376  resulted  in  a more  accurate  and  complete  report.  No  advan- 
tage could  be  found  in  using  either  the  map  display  or  the  map  display 
with  time  compression.  Moreover,  no  significant  differences  could  be 
found  between  the  two  types  of  map  displays.  These  results  hold  both 
for  false  alarms  and  for  accuracy  of  report,  regardless  of  activity 
level.  With  regard  to  completeness,  however,  use  of  any  of  the  three 
displays  resulted  in  equally  good  performance  during  low- target  activity. 

Under  high-target  activity,  use  of  the  RO  376  resulted  in  more  correct  ) 

target  detection  than  did  the  other  displays. 

The  superiority  of  the  RO  376  was  found  for  both  levels  of  experi- 
ence represented  in  this  experiment.  However,  experience  as  measured 
here  did  not  relate  to  performance,  thereby  casting  doubt  on  the  meaning 
of  the  term.  Different  results  might  occur  between  the  groups  used  in 
this  experiment  and  a group  just  graduating  from  training. 
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The  other  variables — Time  Periods,  Scenarios,  Order — were  included 
as  control  variables.  Significant  effects  generally  were  found  for  these 
variables.  Thus,  in  future  experimentation,  these  variables  must  be 
control led . 

The  objective  measures  of  performance  used  in  the  experiment — 
accuracy  and  completeness — assist  the  commander  in  assessing  the  use- 
fulness of  intelligence  generated  by  experienced  personnel  using  the 
operational  RO  376  unattended  ground  sensor  display  for  situations  simi- 
lar to  those  used  in  this  research.  The  performance  accuracy  of  80% 
and  report  completeness  of  88%  attained  with  use  of  the  RO  376  indicate 
that  this  system  is  providing  valid  information.  There  is,  however,  room 
for  improvement.  In  subseguent  experiments  of  this  series,  other  varia- 
tions in  conditions  under  which  the  RO  376  is  used  should  be  included. 
Indexes  of  performance  may  vary,  depending  on  factors  in  the  operational 
situations  such  as  operator  experience,  sensor  type  and  mix,  sensor  de- 
ployment, target  size,  number  of  targets,  target  speed,  soil  type,  noise 
factors,  etc.  Combining  these  results  with  those  of  other  controlled 
studies  will  result  in  a more  accurate  estimate  of  the  capabilities  of 
UGS  systems. 

A possible  factor  mitigating  the  above  conclusions  was  the  Pro- 
ficiency factor.  The  operators  were  experienced  in  the  use  of  the  RO  376, 
but  had  not  actually  used  the  map  display  prior  to  the  training  given 
for  the  experiment.  Although  they  were  given  4 hours  training  and  prac- 
tice, it  is  not  known  if  this  was  sufficient  to  increase  their  proficiency 
in  using  these  devices  to  the  level  of  the  RO  376.  Analyses  comparing 
the  Day  1 to  Day  2 performance  showed  no  differences,  indicating  that  no 
learning  took  place  in  that  brief  period.  Actually,  more  wrongs  were 
found  for  Day  2 than  for  Day  1,  which  contraindicates  learning.  It 
is  still  possible,  however,  that  experience  could  be  confounding  the 
Display  variable. 

A limiting  factor  to  the  feasibility  of  generalizing  results  is  the 
sampling  used  in  designing  the  scenarios.  The  scenarios  did  contain 
operationally  valid  activations  by  both  personnel  and  vehicles  plus  some 
of  the  usual  background  noise  associated  with  a battlefield  situation. 

The  sensors  were  operational  sensors,  employed  in  strings  similar  to 
those  used  in  Viet  Nam,  and  the  operators  were  reguired  to  report  normal 
target  information.  The  sensors  were  not  deployed  as  they  would  be  for 
the  area  intrusion  concept,  however,  in  which  sensors  are  deployed  in 
checkerboard  fashion.  This  type  of  deployment  may  present  more  problems 
to  RO  376  operators  than  does  deployment  in  strings.  The  typical  stair- 
step pattern  used  by  operators  to  detect  the  presence  of  targets  will  no 
longer  be  as  obvious.  Under  the  area  intrusion  concept,  the  path  of  a 
target  may  be  more  obvious  on  a map  display  than  on  the  RO  376,  espe- 
cially having  time  compression  capability  to  allow  the  operator  a quick 
review.  Also,  the  patterns  of  several  targets  through  an  area  may  be 
much  easier  to  determine  using  a map  display  with  time  compression. 
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Imp  1 i cat  ions  of  the  Find  i ngs 

1.  When  sensors  in  strings  are  used,  the  operational  RO  37f>  event 
recorder  or  similar  device  should  be  used  rather  than  the  situation  map 
display  or  situation  map  display  with  time  compression. 

2.  Results  showed  the  necessity  of  controlling  certain  variables — 
level  of  target  activity,  order  of  presentation,  and  scenario  composi- 
tion— in  subsequent  tests  of  displays  for  more  refined  evaluation. 

3.  An  additional  study  using  sensors  deployed  for  area  intrusion 
detection  should  be  accomplished  before  discarding  map  displays  or  time 
compression  techniques  as  useful  devices  for  the  field. 


- 

. 
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APPENDIX  A 


BRIEFING  FOR  EXPERIMENT  TO  EVALUATE  IJOS  DISPLAYS 


I am  Mr.  Hilligoss,  a research  psychologist  with  the  U.S.  Army  Re- 
search Institute  in  Arlington,  Virginia.  Mr.  Lavicka,  Mr.  Parker,  and 
Mr.  Thorpe  are  electronics  experts  from  the  U.S.  Army  Electronics  Command, 
Fort  Monmouth,  N.J. 

Our  purpose  in  coming  to  this  command  for  this  week  and  a half  is 
to  evaluate,  with  your  assistance,  three  types  of  unattended  ground  sen- 
sor displays.  The  Navy  has  agreed  to  cooperate  for  several  reasons  but 
mainly  because  they  are  just  as  interested  in  UGS  displays  as  i the 
Army.  In  order  to  objectively  evaluate  these  displays,  we  need  you,  as 
trained  specialists,  to  use  each  of  them  so  that  we  can  determine  which 
display  results  in  the  best  performance,  that  is,  in  fewer  false  alarms 
and  a greater  number  of  correct  reports  of  enemy  activity.  Thus,  you 
are  helping  to  determine  what  equipment  you  may  use  in  the  future. 

We  have  taped  actual  sensor  activations  in  the  field  at  Fort  Hood, 
Tex.,  under  simulated  battlefield  conditions  including  typical  noise, 
such  as  artillery,  helicopters,  and  wind.  These  tapes  will  be  played 
back  to  activate  the  three  display  devices.  You  will  interpret  the  dis- 
( plays  and  make  a report  much  as  you  would  operationally,  but  using  our 

refjort  forms.  Since  we  know  where  and  when  target  activations  actually 
occurred,  we  can  score  your  reports  for  accuracy  and  thereby  determine 
which  is  the  best  display. 

You  will  work  on  this  experiment  for  about  2 days.  During  that 
time,  you  will  be  given  training  on  the  new  display  equipment  and  a 
practical  exercise  to  familiarize  you  with  the  test  situation.  Then  you 
will  work  with  each  type  of  display  for  2 hours,  with  appropriate  breaks, 
lunch,  etc.  If  you  cannot  be  here  during  the  scheduled  time,  tell  us 
now  so  we  can  reschedule  you.  You  must  be  here  for  all  scheduled  times 
or  we  cannot  use  your  results.  Here  is  your  schedule  for  the  experiment. 
(Note:  Make  up  is  possible  but  do  not  advertise.) 

Now  about  the  displays.  The  first,  the  RO  376  event  recorder, 
you  have  trained  on  and  have  worked  with  in  the  field.  This  will  be 
the  standard  display  because  it  is  the  present  operational  device. 

Your  reports  using  this  device  will  be  compared  to  your  results  with 
the  other  devices.  Training  on  the  RO  376  will  bo  minimal  since  this 
device  should  be  familiar  to  you. 

The  second  display,  the  situation  map  display,  is  simply  a map 
with  lights  indicating  the  location  of  the  sensors.  Each  sensor  acti- 
vation causes  the  corresponding  light  to  blink  for  5 seconds.  Thus  you 
can  see  the  progression  of  the  enemy  on  the  map  by  which  light  is  blink- 
ing. Finding  location  and  direction  is  easy,  of  course,  but  gettinq 
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speed  may  be  a little  more  difficult.  Naturally,  we  will  give  you  prac- 
tice on  this  device  and  you  will  not  be  required  to  maintain  or  set  it 
up — just  interpret ! 

The  third  display  is  identical  to  the  second  except  that  you  have  a 
review  or  playback  capability.  With  the  situation  map  display,  you  have 
no  permanent  record  or  way  of  checking  on  your  decisions  as  you  do  wi th 
the  HO  376.  However,  if,  in  the  field,  there  were  a tape  recorder  to 
record  the  activations,  you  could  play  them  back  if  you  wished  to  see  the 
activations  a second  time.  In  addition,  if  you  play  the  recording  back 
at  8 times  normal  speed,  you  would,  in  a sense,  compress  time,  and  be 
able  to  review  more  quickly.  This  also  might  help  you  see  a pattern  more 
easily.  This,  then,  is  your  third  display — a situation  map  display  that 
allows  you  to  play  back  what  has  happened  in  the  past,  at  8 times  normal 
speed.  You  will  notice  that  during  the  review  period  the  light  will  not 
blink.  It  comes  on  for  one-half  second  for  each  activation.  (Pause) 

As  an  additional  condition  of  this  display,  every  half  hour  you  will 
have  a required  review  of  what  has  occurred  during  the  previous  half 
hour  but  in  8x  normal  speed.  This  forced  review  is  another  approach  we 
are  trying  out. 

There  are,  of  course,  many  display  ideas  and  procedures  we  could, 
and  will,  try  out.  This  is  only  the  first  of  a series  of  experiments 
in  the  area.  When  you  are  finished,  please  give  us  your  ideas. 

Your  task,  then,  with  this  third  display  will  be  to  interpret  the 
blinking  lights,  which  represent  activations  in  the  field,  on  the  map 
display  exactly  like  the  second  display.  However,  if  you  wish  to  review 
what  you  thought  to  be  a target  pattern  you  would  so  indicate.  We  would 
stop  the  clock  and  replay  at  8 times  normal  speed  the  last  few  minutes 
of  activations.  For  the  purposes  of  this  experiment,  we  would  pretend 
no  time  elapsed  during  this  playback.  When  the  review  was  over,  you 
would  once  again  view  and  display  in  normal  time.  Then,  after  30  minutes 
(of  clock  time),  you  would  have  a forced  review  period  of  about  4 minutes. 

Before  we  explain  how  to  fill  out  the  target  report  form,  are  there 
any  questions?  (Pause) 

Here  are  the  target  report  form  and  map  sketches  of  the  areas  of 
concern.  We  are  using  map  sketches  to  simplify  the  experiment.  Note 
the  first  column  on  your  report  form.  We  want  to  know  when  you  first 
thought  a target  might  be  present.  Target  indication  should  be  given 
in  terms  of  the  time  the  target  activated  the  second  sensor  in  the  string. 
Using  the  second  sensor  should  give  you  enough  time  after  the  first  sen- 
sor to  decide  whether  a target  is  present.  If  this  does  not  give  you 
enough  time,  then  give  us  the  time  the  target  activated  the  third  sensor. 
But  be  sure  to  indicate  in  the  remarks  column  on  your  target  report  form 
that  this  is  the  third  sensor. 
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Indicate  in  column  2 which  sensors  are  activated.  These  are  the 
same  as  pen  numbers  or  the  numbers  by  the  lights. 

In  column  3,  put  down  the  time  when  you  finally  decide  a target  is 
present — this  may  be  the  same  time  as  the  first  column  or  some  time  later. 

Next,  determine  the  velocity,  using  a table  we  will  provide.  You 
can  accurately  measure  the  time  on  the  RO  376.  However,  with  the  map 
displays  you  will  have  to  use  the  clocks  to  determine  the  time  interval 
between  the  activations  of  different  sensors  in  a string. 

In  the  target  type  column,  you  will  write  either  a V for  vehicle  or 
P for  personnel.  This  probably  will  be  determined  from  the  velocity  col- 
umn and  your  experience  as  an  operator. 

In  the  next  column,  give  your  estimate  of  the  number  of  targets 
present.  This  will  be  extremely  difficult  but  do  the  best  you  can. 

In  the  next  column,  we  want  you  to  try  to  give  us  your  confidence 
that  there  is  a target  present.  This  is  an  additional  piece  of  intelli- 
gence information  which  has  been  found  very  useful  in  other  areas  of  the 
intelligence  field.  We  wish  to  determine  how  well  this  judgment  can  be 
made  and  how  the  use  of  the  different  displays  affects  your  estimate. 

Use  only  the  numbers  100,  90,  80,  70,  60,  50,  40,  30,  20,  and  10. 

If  you  put  down  90,  this  means  you  are  quite  sure  there  is  a target 
there,  but  you  expect  to  be  right  only  90  times  out  of  100;  if  you  put 
50  down,  this  means  that  you  estimate  that  only  half  the  reports  so 
marked  will  be  right.  If  you  put  down  10,  then  only  10%  of  the  time 
will  you  be  right.  Naturally,  for  all  those  targets  where  you  put  100 
down,  you  are  estimating  that  you  will  always  be  right.  Use  the  next 
column  as  is  necessary. 

If  you  wish  to  delete  or  change  anything  draw  a line  through  it  and 
write  above  it  either  "omit"  or  the  change.  Please  do  not  erase. 

Are  there  any  questions?  (Pause) 

What  you  will  be  seeing  was  recorded  during  maneuvers  at  Fort  Hood, 
Tex.  There  will  be  varying  numbers  of  personnel  or  vehicles  activating 
the  sensors.  In  addition,  there  will  be  the  normal  noise  due  to  weather, 
wind,  and  so  forth,  and  noise  from  artillery  and  helicopters.  Do  not 
report  artillery,  helicopters,  etc.,  only  personnel  or  vehicles.  You 
should  note  on  our  map  simulations,  that  the  intervals  in  different  sen- 
sor strings  are  different.  Be  careful  when  computing  the  velocity  in- 
formation. Always  refer  back  to  the  map. 

Each  of  the  three  scenarios  will  start  at  12  midnight  (0000  hours) , 
and  will  run  for  a period  of  approximately  2 hours.  Please  remove  your 
wristwatch  and  put  it  in  your  pocket.  Use  only  the  clocks  supplied  in 
the  experiment. 
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I would  like  to  emphasize  that  we  are  not  qivinq  you  a test  to  see 
how  good  an  operator  you  are.  This  experiment  is  for  the  purpose  of  de- 
termining which  display  is  best  for  extracting  target  information.  The 
Army  is  not  interested  in  how  good  you  are  as  an  operator.  However,  you 
and  your  superiors  are  interested  in  how  good  you  are.  I am  sure  they 
will  not  base  the  next  promotion  on  how  well  you  do  on  6 hours  of  Army 
practical  exercises.  Still,  these  activations  are  actual  activations 
recorded  in  the  field  and  your  accuracy  in  interpreting  is  one  indica- 
tion or  example  of  what  you  can  do.  You  will  be  able  to  compare  what 
you  do  to  what  others  do  as  a group.  You  will  be  able  to  get  your  score 
and  the  group  average  from  your  commanding  officer.  He  will  be  able  to 
assess  you  objectively  against  the  others  on  this  one  sample  of  one  of 
your  duties.  However , there  are  no  standards  of  performance — even  if 
you  do  worse  than  everyone  else,  you  still  could  be  a crjmpetent  operator. 


I 


APPENDIX  B 


T PAINING  OF  SUBJECTS  IN  THE  USE  OF  MAP  DISPLAYS 


A scenario  was  prepared  similar  to  the  three  scenarios  that  the 
examinees  were  to  see  in  evaluating}  the  three  readout  displays.  This 
scenario  was  in  two  sections:  a training  section  and  a practical  exer- 
cise (PE)  section.  The  training  section  consisted  of  four  30-minute 
parts,  each  containing  personnel  and  tracked  vehicles.  The  PE  followed 
the  same  format  with  different  test  items  and  followed  the  training 
after  a break. 

After  the  briefing  given  in  Appendix  A,  the  following  procedure  was 
presented  to  the  subjects  and  repeated  if  necessary  during  the  training 
and  practical  exercise  sessions: 


Proce dure  for  Map  Display 

1.  Watch  for  blinking  lights. 

2.  If  you  believe  a sensor  might  have  been  activated  by  a target, 
flag  it  with  a paper  clip  or  grease  pencil. 

3.  Continue  to  view  all  lights. 

4.  When  the  second  sensor  in  the  string  is  activated,  note  time 
on  answer  sheet  if  you  believe  a target  might  be  present. 

5.  Continue  viewing  with  special  attention  to  activated  string. 

6.  If  you  decide  a target  is  present,  fill  out  form.  If  you  de- 
cide no  target  is  present,  write  "omit"  to  right  of  whore  you  put  time 
in  first  column. 

[Add  for  map  display  plus  time  compression] 

7.  If  you  wish  to  review,  tell  the  monitor  the  amount  of  time 
you  wish  to  review  (i.e.,  2,  4,  6,  8 minutes).  Monitor  will  stop  the 
clock,  reverse  the  tape  the  amount  of  time  requested  and  will  play  back 
the  tape  at  8x  normal  time  to  the  spot  in  the  tape  where  you  requested 
the  review.  He  will  then  start  the  clock  and  tape  at  normal  time  and 
you  will  continue  as  before.  This  is  a simulation  of  what  would  happen 
in  the  field.  Other  equipment  would  be  designed  to  handle  this  function 
if  we  find  it  to  be  helpful. 

8.  Note  that  you  will  have  no  clock  during  the  review  period.  If 
you  wish  to  record  time  during  this  period,  that  is,  report  a possible 
target,  say  loudly  to  the  operator  "time."  He  will  tell  you  the  tape 
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recorder  counter  numbers  which  you  will  use  as  time  in  the  first  or  third 
column  on  your  answer  sheet.  We  will  later  convert  these  numbers  to  the 
real  time.  Finally,  you  also  may  tell  the  operator  to  stop  if  you  wish 
to  fill  in  the  answer  sheet  and  then  continue  to  view  at  8x  normal  time. 

To  conserve  training  time,  the  group  monitored  the  display  together. 
The  senior  member  of  each  team  was  chosen  as  the  team  leader;  one  person 
had  the  responsibility  of  computing  ground  speed,  the  other  completed  the 
answer  sheet.  In  this  way,  gathered  around  one  readout  device,  there  was 
interaction  among  the  examinees  and  answers  given  to  questions  by  one 
benefited  all. 

A participant  could  ask  for  a replay  of  the  magnetic  tape  in  com- 
pressed time  for  any  portion (s)  of  the  scenario  that  person  wanted  to 
view  again.  After  completion  of  each  half  hour  section  of  the  training 
tape,  that  section  was  rerun  in  its  entirety  in  compressed  time  to  insure 
training  in  the  use  of  compressed  time.  After  the  2-hour  training  tape 
was  completed,  a discussion  followed  to  allow  the  experimenters  to  empha- 
size the  important  points  to  remember  and  also  to  give  additional  time 
for  questions  and  answers. 

Following  a break,  the  examinees  took  the  exercise.  The  team  mem- 
bers took  turns  in  interpreting  the  display,  estimating  speed,  and  com- 
pleting the  report  form.  The  proctors  gave  assistance  in  procedure  but 
did  not  assist  in  the  interpretation. 

All  of  the  examinees  except  two  had  had  experience  in  interpreting 
event  recorders  in  field  exercises.  Additional  training  on  the  RO  376 
by  an  experienced  IJflS  operator  was  given  to  the  two  men  before  the  ex- 
periment started. 
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1 AFHRL  (FTI  Wilham  AFB 
1 AFHRL  (TT)  Lowry  AFB 

1 AFHRL  (AS)  WPAFB,  OH 

2 AFHRL  IDOJZ)  Brooks  AFB 

1 AFHRL  I DO  JN ) Lackland  AFB 
1 HQUSAF  (INYSD) 

1 HQUSAF  IDPXXA) 

1 AFVTG  (RD)  Randolph  AFB 

3 AMRL  (HE)  WPAFB.  OH 

2 AF  Inst  of  Tech,  WPAFB.  OH,  ATTN:  ENE/SL 
1 ATC  (XPTD)  Randolph  AFB 

1 USAF  AeroMed  Lib,  Brooks  AFB  (SUL-4),  ATTN:  DOC  SEC 
1 AFOSR  (NL).  Arlington 

1 AF  Log  Cmd.  McClellan  AFB.  ATTN.  ALC/DPCRB 

1 Air  Force  Academy,  CO,  ATTN:  Dept  of  Bel  Sen 
5 NavPers  & Dev  Ctr,  San  Diego 

2 Navy  Med  Neuropsychiatric  Rsch  Unit.  San  Diego 
1 Nav  Electronic  Lab,  San  Diego,  ATTN:  Res  Lab 

1 Nav  TrngCen,  San  Diego,  ATTN:  Code  9000— Lib 
1 NavPostGraSch,  Monterey,  ATTN:  Code  55Aa 
1 NavPostGraSch,  Monterey.  ATTN:  Code  2124 
1 NavTrngEquipCtr,  Orlando,  ATTN:  Tech  Lib 
1 US  Dept  of  Labor,  DC,  ATTN:  Manpower  Admin 
1 US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1 Nat  Bur  of  Standards,  DC.  ATTN:  Computer  Info  Section 
1 Nat  Clearing  House  for  MH-Info,  Rockvilfe 
1 Denver  Federal  Ctr.  Lakewood.  ATTN:  BLM 
12  Defense  Documentation  Center 

4 Dir  Psych.  Army  Hq.  Russell  Ofcs,  Canberra 

1 Scientific  Advsr.  Mil  Bd,  Army  Hq.  Russell  Ofcs,  Canberra 

1 Mil  and  Air  Attache.  Austrian  Embassy 

? Centre  de  Recherche  Des  Facteurs,  Humaine  de  la  Defense 
Nationale.  Brussels 

2 Canadian  Joint  Staff  Washington 

1 C/Air  Staff,  Royal  Canadian  AF,  ATTN:  Pers  Std  Anal  Br 

3 Chief.  Canadian  Def  Rsch  Staff.  ATTN:  C/CRDS(W) 

4 British  Def  Staff,  British  Embassy,  Washington 
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